Conversation
aaron-kaplan
left a comment
There was a problem hiding this comment.
Possibly a faster approach: keep the outer loop (window length), but replace the inner loop with a rolling sum. Instead of a single result array count, keep two: count and length. Then after calculating the rolling sum but before updating count, zero out new_event for windows that overlap a window that has already been counted.
I worked on this but as of now I don't see a way to do it all. I can see how not to count bigger windows that overlap with smaller windows previously identified via the window loop, but I don't see how to disqualify overlapping windows of same size. E.g. say you have 4 days with a, b, c, d values. All the size-w windows (ab, bc, cd) meet the criterion, bc is disqualified because b already contributing to ab, but that makes cd a valid count, but I can't know that before I tell bc is not valid. I don't see how to do that without some looping incrementation. |
|
@aaron-kaplan , it doesn't seem possible to remove a reviewer so please so whatever you want to do here and in the future I won't include you if data ingestion or app set up is not involved. |
|
@remicousin This is the stress test I mentioned (the results). 1D means a single point with a time series of X days (1 year, 10 years, 30 years). 3D includes X and Y (a grid of approximately 27×60, which is more/less what the U.S. has at one-degree resolution). There are two tabs: one for 3-day windows and another for 8-day windows. The results in red correspond to your function. Please ignore the blue ones, because until I manage to make them perform the same associations as your function, they are not relevant. note that I ran these tests on my laptop, which has the following specification: |
I tried to think of a vectorial solution but couldn't. Probably because there is a "memory" aspect of the analysis: once a day is part of an extreme event, it can't be part of another one.